1. Field of the Invention
This invention relates to the field of video image filtering. More particularly, this invention relates to the field of video image filtering using multi-tap filters, such as finite impulse response filters.
2. Description of the Prior Art
It is known to carry out video image filtering using multi-tap filters for a variety of purposes, e.g. to remove undesirable artifacts from an image or as part of a data compression system using sub band filtering. Filtering video image data with multi-tap filters is subject to problems resulting from the finite extent of the image being filtered. FIG. 1 of the accompanying drawings illustrates this problem.
Blocks of sample data 2, 4 corresponding to respective vertical and horizontal rasters through the image are fed to corresponding finite impulse response filters 6, 8. The finite impulse response filters 6, 8 have five taps. The filtered data value corresponding to a given sample point within the input data is generated by the adders 10, 12 when that given input sample value is at the centre tap of the filter.
In the example illustrated, the first sample value of each of the rasters is at the centre tap. Whilst the upstream taps of the filter are loaded with valid data, the downstream taps carry invalid data read from the blanking period of the input video signal. The consequence of this is that until the filter is fully loaded with valid data, the filtered data values produced by the adders 10, 12 are distorted. When the filtered data values generated in this way are subsequently used to produce an image 14 for display, the result of this distortion is visible around the edges of the produced image.
In order to counter this problem it has been proposed that the rasters/blocks of input video data should be extended prior to being fed into the multi-tap filters. FIG. 2 of the accompanying drawings illustrates a number of different extension strategies that may be used.
End extension repeats the first and last values at either end of the active data. Circular extension prefixes the active data with the sequence of samples from the end of the active data and suffixes the active data with the sequence of samples from the beginning of the active data. Circular extension is intended to give a form of limited periodicity to the data. Symmetric extension adds a mirror image copy of the first part of the active data to the beginning of the active data and a mirror image copy of the last part of the active data to the end of the active data.
Whilst the above has been described in terms of rasters/blocks of raw image data, it will be appreciated that video image filtering is often a multi-stage process and that the data that is input to a multi-tap filter may already have undergone processing which alters the nature of what the active data within that raster/block actually represents. An example of this is shown in FIG. 3 of the accompanying drawings. FIG. 3 shows a raster of data which has already undergone spatial frequency sub-band filtering. The low frequency components of the original data are collected together in a block at the beginning of the raster and the high frequency components of the original data are collected together in a block at the end of the raster.
The low frequency components and high frequency components 16 are the active data within the raster and are separated by blanking periods. Into these blanking periods extension data 18, 20 is added. The extension data 18 is derived from the low frequency component active data and the extension data 20 is derived from the high frequency component active data. Each section of valid active data is thus extended so that distortion upon subsequent filtering with a multi-tap filter is reduced.
It will be appreciated that the sub band filtering transforms the raw image into the spatial frequency domain, but reducing distortion in this transformed image per se is not the main priority. The transformation is carried out as part of a data compression technique. When the data is decompressed and transformed back into the spatial domain, the most important factor is the degree of distortion in the reconstructed image. In most cases a reduction of distortion in the transformed image will result in a reduction of distortion in the reconstructed image, but it is important to keep in mind the priorities of the complete system.
FIG. 4 of the accompanying drawing illustrates how the use of the technique of extending the data results in a reduction in distortion at the edges. The system illustrated in FIG. 4 is essentially the same as that illustrated in FIG. 1 with the addition of the data extension units 20, 22. The data extension units 20, 22 carry out the extension of the rasters of data as illustrated in FIGS. 2 and 3. In the case illustrated, symmetric extension is employed. By extending the raster line of data, the filters 6, 8 are effectively fully preloaded with valid data by the time the first true sample (sample 1) arrives at the centre tap. In this way, the edge distortion is pushed away from data that will be used in the subsequently displayed image 14.
The edge distortion still occurs at the edges of the extended rasters of data, but is confined within the extended portion. Since this extended portion does not in fact represent true data and will subsequently be discarded prior to display, the effect of edge distortion within the valid data is reduced.
From the foregoing it will be seen that the technique of edge extension is highly effective in reducing edge distortion. If one is carrying out image filtering on a non-real time basis (e.g. still image processing in a computer system), then the additional overhead of the step of extending the rasters of data slightly increases the time taken to process the image. The implementation of the edge extension technique in a real time video system is not straightforward.
In any video image processing system it is desirable for the purposes of reducing cost and complexity to have a single clock/data sample rate throughout the system. Running any part of the system at a higher rate introduces problems of maintaining synchronisation and also involves the use of more expensive higher speed components.
Whilst the filters 6, 8 of FIGS. 1 and 4 have been shown with only 5 taps, a production filter would have far more taps, e.g. a 32 tap filter would not be unusual. With these much longer filters, longer extension blocks must be added to the rasters of data in order to effectively pre-load the filter prior to the first valid sample arriving at the centre of the filter. Thus the problem arises of adding long extension blocks with hardware that is only running at the base data sample rate.